Skip to content

fix: use SetFunctionCallNameString when forcing a specific tool (3 sites) - #9526

Merged
mudler merged 3 commits into
mudler:masterfrom
Anai-Guo:fix/tool_choice-wrong-setter
Apr 24, 2026
Merged

fix: use SetFunctionCallNameString when forcing a specific tool (3 sites)#9526
mudler merged 3 commits into
mudler:masterfrom
Anai-Guo:fix/tool_choice-wrong-setter

Conversation

@Anai-Guo

Copy link
Copy Markdown
Contributor

Summary

Follows up on #9509 (merged) which fixed the same bug in MergeOpenResponsesConfig.

Three additional call sites were using SetFunctionCallString(name) (which writes the mode field, e.g. "none"/"auto"/"required") when they should be using SetFunctionCallNameString(name) (which writes the specific function name field read by ShouldCallSpecificFunction / FunctionToCall).

Net effect of the original code: when a caller sends tool_choice: {type: "function", function: {name: "my_tool"}} via these three endpoints, the grammar-based forcing is silently never engaged because the name ends up in the wrong config field.

Sites fixed

File Line Endpoint
core/http/endpoints/anthropic/messages.go 883 Anthropic /v1/messages
core/http/endpoints/openai/realtime_model.go 171 OpenAI Realtime
core/http/endpoints/openresponses/responses.go 776 /v1/responses (non-middleware path)

In each case the change is:

- cfg.SetFunctionCallString(name)
+ cfg.SetFunctionCallNameString(name)

Tracked in issue #9508.

Test plan

  • Send tool_choice: {"type": "function", "function": {"name": "my_tool"}} via Anthropic /v1/messages endpoint and verify grammar-based forcing engages
  • Same via OpenAI Realtime endpoint
  • Same via /v1/responses endpoint

🤖 Generated with Claude Code

@mudler
mudler merged commit 5e062b4 into mudler:master Apr 24, 2026
1 check passed
Anai-Guo added a commit to Anai-Guo/LocalAI that referenced this pull request Apr 25, 2026
Follows up on mudler#9526 (the 3-site setter fix) by addressing the remaining
clause in mudler#9508 — string mode and OpenAI-spec specific-function shape both
silently failed in the /v1/chat/completions parsing path.
@localai-bot localai-bot added the bug Something isn't working label May 9, 2026
mudler pushed a commit to Anai-Guo/LocalAI that referenced this pull request May 13, 2026
Follows up on mudler#9526 (the 3-site setter fix) by addressing the remaining
clause in mudler#9508 — string mode and OpenAI-spec specific-function shape both
silently failed in the /v1/chat/completions parsing path.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
mudler added a commit that referenced this pull request May 13, 2026
#9559)

* fix(middleware): parse OpenAI-spec tool_choice in /v1/chat/completions

Follows up on #9526 (the 3-site setter fix) by addressing the remaining
clause in #9508 — string mode and OpenAI-spec specific-function shape both
silently failed in the /v1/chat/completions parsing path.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

* fix(middleware): restore LF endings and cover tool_choice parsing with specs

The previous commit on this branch saved core/http/middleware/request.go
with CRLF line endings, ballooning the diff against master to 684 / 651
for what is in reality a ~50-line parsing change. Restore LF (matches
.editorconfig end_of_line = lf).

Add 11 Ginkgo specs under "SetModelAndConfig tool_choice parsing
(chat completions)" that parallel the existing MergeOpenResponsesConfig
specs from #9509. They drive the full middleware chain (SetModelAndConfig
+ SetOpenAIRequest) and assert:

  * "required"  -> ShouldUseFunctions=true, no specific name
  * "none"      -> ShouldUseFunctions=false (tools disabled per OpenAI spec)
  * "auto"      -> default, tools available, no specific name
  * {type:function, function:{name:X}}  (spec)    -> X is forced
  * {type:function, name:X}             (legacy)  -> X is forced
  * nested wins when both forms are present
  * malformed shapes (no type, wrong type, no name, empty name) are no-ops

Update the inline comment on the string case to describe the actual
mechanism: "none" reaches SetFunctionCallString("none") downstream and
is then honored by ShouldUseFunctions() returning false. Before this PR
json.Unmarshal([]byte("none"), &functions.Tool{}) failed silently, so
"none" was ignored - making "none" actually work is a real behavior fix
this PR brings.

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:opus-4-7 [Claude Code]

* fix(middleware): preserve pre-#9559 support for JSON-string-encoded tool_choice

Some non-spec clients send tool_choice as a JSON-encoded string of an
object form, e.g. "{\"type\":\"function\",\"function\":{\"name\":\"X\"}}".
The pre-#9559 code accepted this by accident: its case string: branch
ran json.Unmarshal([]byte(content), &functions.Tool{}), which succeeded
for that double-encoded shape even though it failed for the legitimate
plain string modes "auto" / "none" / "required".

The first version of this PR routed every string straight to
SetFunctionCallString as a mode, which fixed the plain-string cases but
silently regressed the double-encoded one (funcs.Select("{...}") returns
nothing). Restore the fallback: when a string looks like a JSON object,
try parsing it as a tool_choice map first; fall through to mode-string
handling only when no usable name comes out.

Factor the map-name extraction into a small helper
(extractToolChoiceFunctionName) so the string-fallback and the regular
map case go through identical code, and accept both the OpenAI-spec
nested shape and the legacy/Anthropic flat shape from either entry
point.

Add 3 Ginkgo specs covering the double-encoded case (nested form, legacy
form, and the fall-through when the JSON has no usable name).

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Assisted-by: Claude:opus-4-7 [Claude Code]

* test(middleware): silence errcheck on AfterEach os.RemoveAll

The new tool_choice parsing tests added a second AfterEach that calls
os.RemoveAll(modelDir) without checking the error; errcheck flagged it.
Suppress with the standard _ = idiom. The pre-existing AfterEach on the
earlier Describe still elides the check the same way it did before -
leaving that untouched to keep this commit minimal.

Assisted-by: Claude:opus-4-7 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

---------

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants